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■ Abstract 

^ ■ Feedback control (based on the quantum continuous measurement) of quantum systems inevitably suffers from 
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estimation delays. In this paper we give a delay-dependent stability criterion for a wide class of nonlinear stochastic 
systems including quantum spin systems. We utilize a semi- algebraic problem approach to incorporate the structure 
of density matrices. To show the effectiveness of the result, we derive a globally stabilizing control law for a quantum 
spin- 1/2 systems in the face of feedback delays. 
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CN ■ I. Introduction 

0^ , Quantum systems substantially differ from classical (i.e., non-quantum) systems in that state variables are 

CO 

£C) ■ represented by noncommutative operators acting on a Hilbert space; see e.g., [1]. Such noncommutativity imposes 

some critical constraints on the structure of a quantum controller. This makes it difficult to analyze/synthesize 

ON 

' feedback control systems for quantum systems. However, quantum filtering theory [2], [3], [4], [5] has clarified that 

i> : 

, a number of quantum control problems can be formulated and solved within the framework of standard classical 

stochastic control theory [6], [7], [8], [9], [10], [11], [12], [13], [14], [15]. 
■ A brief description of the filter-based approach to quantum control is as follows. The plant dynamics are given 

by a quantum stochastic differential equation, where the state is a noncommutative random variable [16]. The 
dynamics are partially monitored by means of a continuous measurement that allows us to construct an estimator 
of the plant variables. The resulting filter is a classical stochastic differential equation called the Belavkin equation 
or stochastic master equation. Our objective is to synthesize an effective controller such that the filter shows a 
desirable behavior. 

For this problem, two types of control law have been proposed. The first one is a simple proportional feedback of 
the output signal. The second one is a feedback of the estimate of the plant variables, which we call the filter-based 
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controller. A more detailed description of these two controllers will be given in the next section, but we here note 
that, for the implementation of the filter-based controller, a non-negligible computation time is required to process 
the estimation [17]. Therefore, from a practical point of view, a filter-based feedback controller should be considered 
taking the feedback delay into explicit account. For example, Steck et al. have numerically studied the issue of 
delay in the case of feedback stabilization of atomic motion [18]. However, to the authors' best knowledge, there 
have been no theoretical means to perform such investigations in the quantum case. 

In this paper we study the effect of the delay in quantum systems with the full use of several techniques for 
analyzing stability of stochastic delay differential systems; see e.g., [19], [20], [21] and references therein. In 
particular, we focus on the control problem of a quantum spin system, which has also been studied in [8], [10], 
[13]. This system is very important, since it is one of the most basic components in quantum information processing 
[22]. 

This paper is organized as follows. Section II reviews quantum filtering and control. In particular, we discuss 
delay in this feedback control scheme. Section III is the main part of this paper. Theorem 1 gives a delay-dependent 
stability criterion for a class of nonlinear stochastic systems including some quantum spin systems. The effectiveness 
of the result is then verified by deriving a stabilizing controller for the spin- 1/2 particle case. 

Notation For z G R n and M G M nxn , \\z\\ 2 M := z J Mz. The subscript is omitted when M is the identity 
matrix. A function F : D —> R is said to be negative (resp. positive) in D if F(z) < (resp. F(z) > 0) for any 
z G D. A subset C in R n is said to be semi-algebraic if 

C := {x G R n : Pi(x) < 0, i = 1, 2, • • • , 1} 

with polynomials pi . 

Let Cq be the set of C-valued uniformly continuous functions on [—ft, 0]. This is a Banach space equipped with 
H^llcf := sn Poe[-h,o] II^W Ik- Given a probability measure, the probability and expectation are denoted by P and 
E. We say an event Q occurs almost surely if P{^} = 1. If it exists, the infinitesimal generator of a function V 

along a Markov process x t is denoted by srfV i.e., s4V{x) := lim where W represents the expectation 

with respect to paths which start at xq = x; see [20], [19], [21] for a formula. 

II. Control scheme based on Quantum filtering 

A. Quantum filtering 

We here provide a brief summary of quantum filtering theory [2], [3], [4]. For a more detailed description, see 
[5]. 

In the framework of quantum filtering, a plant dynamics is described in a similar form to a general classical 
stochastic differential equation. For example, when using a homodyne detector [23], a single state variable X t 
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satisfies 

dX t = f(X t )dt + g(X t )dW t , 

dY t = (h(X t ) + h(X t )*)dt + dV u (1) 

where /, g, and h are smooth functions with specific structures. However, unlike the classical case, the state variable 
Xt, the output Ft, and the stochastic noises Wt, Vt are observables, i.e., Hermitian operators that act on a certain 
Hilbert space (* denotes the self-adjoint operation). Thus, in general they do not commute with each other. Note 
that any noncommutative random variables cannot take their realization values on a same probabilistic space. This 
implies that the classical stochastic control theory is not directly applicable, because we cannot define the conditional 
expectation 7r(X t ) := Ei(X t \y t ), and consequently the optimal filter. Here, y t denotes the set of Y s (0 < s < t). 
Quantum filtering theory identifies systems free from these difficulties, i.e., systems satisfying the nondemolition 
properties [Y 3 ,Y t ] = (Vs,t) and [X t ,Y s ] = (Vs < i), where [A, B] := AB - BA. Fortunately, in many 
important cases, especially in quantum optics, we can build such systems. The filter is then given by 

d7T(X t )=7T(f(X U U t ))dt 

+ (ir(X t h(X t ) + h(X t )*Xt) - 7T(X t )7T(h(X t ) + h(X t )*j) 

x (dY t - 7r(h(X t ) + h(X t )*)dt) . (2) 

Surprisingly, this is the same form as the classical filtering equation except the symmetrized terms. We now introduce 
a density matrix p\ in a finite-dimensional case, it belongs to the convex set 

S:={peC NxN : p = p* >0,trp=l}, (3) 

where N is determined from the system. The statistics of the measurement results of an observable X is completely 
characterized by p. For example, the k-th moment of the outcomes is given by ti(X k p). Thus the conditional 
expectation 7r(X t ) should also be represented in terms of a time-dependent density matrix p t as 7r(X t ) =tr(Xp t ), 
which together with ([2]) leads to the time-evolution of p t . In particular, when the homodyne detection scheme is 
used, the most simple form of it is given by the following stochastic master equation: 

dp t = C*(p t , u t )dt + (Lp t + p t L* - tr(Lp t + p t L*)p^j 

x (dF £ -tr(Lp £ + p £ L*)dt), 
£* (p, u) := i[H, p] + LpL* - h*Lp - l -pL*L. (4) 

Here, H is an observable called Hamiltonian, representing the energy of the system. The measurement operator L 
determines how the system interacts with the measurement apparatus (e.g. a laser field; see Figure 1). 

B. Implementation of filter-based controller 

In a typical situation, the Hamiltonian term is a function of the control input u t ; H = H(u). Our goal is to 
design u t such that the filter of Eq. © has a desirable behavior. Note that, as in the classical case, the last term 
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dw t := dYt — tr (Lpt + ptL*)dt is a classical Wiener increment. This implies that Eq. d4j) is a classical stochastic 
differential equation to which several techniques developed in control theory can be applied. 

The proportional output feedback controller u t = kdY t /dt (k G R is the gain) is often considered [15], [11], 
[12] and was implemented in the experimental setup of spin- squeezing control [24], [25]. On the other hand, note 
that we can compute p t by using the past output sequence {Y s } s < t by Eq. ©. If it is possible to perform this 
computation on-line, we can implement controller of the state feedback form u t = u(p t ), i.e., the filter-based 
controller. With this control the target state is limited to the eigenstates of the measurement operator L unlike the 
proportional feedback case where the target can be to some extent changed flexibly [26], [27], but we can instead 
take much wider variety of designing methods of the filter-based controller. In fact, it has been proven that the 
Lyapunov theory was successfully employed to show the global stability of the filter for some systems [8], [10], 
[13], [28]. Moreover, it is known that the optimal controller for a general type of quantum optimal control problem 
is given by a filter-based controller. This is known as the separation theorem [29]. 

However, in general, the time required to compute p t is not negligible compared to the time-constants associated 
with the dynamics of a nano-mechanical system. In other words, from a practical point of view, p t cannot be used 
to determine u t . In view of this we should consider the delayed feedback control input u t = u(p t - T ), where r > 
denotes the delay length. Note that this formulation is able to handle further delays, for example input delays. 
Such input delays occur because the control input u t must be physically implemented by means of actuators. The 
purpose of this paper is to propose a rigorous methodology for analyzing the behavior of quantum control systems 
in the face of feedback delay. 

III. STABILIZATION OF QUANTUM SPIN SYSTEMS IN THE FACE OF DELAY 

A. The physical model and control problem 

In this section we consider a cold atomic ensemble trapped in an optical cavity [24], [8], [10], [11], [13], as 
depicted in Figure 1. The total angular momentum operator Fi of the atom around the i-axis (i = y, z) is given by 

ci 
-ci c 2 

i 

-cn-2 CAT-1 
-cn-i 
c rn := \/{N — m)m, m = 1, 2, • • • , N — 1, 

F z := ldiag{iV-l,7V-3, 

••• ,-(7V-3),-(7V-l)}, 

where N — 1 represents the number of atoms. The system interacts with a laser field oriented along the z-axis at a 
homodyne-type photo detector, which implies L = F z . The system also interacts with an external magnetic field, 



Fo, := - 
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Fig. 1. Quantum plant and classical controller. The filter needs a finite time r > to compute the control input ut = u(pt-r)- 

which is oriented along the ?/-axis, H(u t ) = u t F y , where the control input u t corresponds to the magnetic field 
strength, which can be modified in time. As a result, the controlled filter equation © becomes 



where r] G (0, 1] represents the measurement efficiency. Note that the Wiener process w t contains the measurement 
data y t . 

Our goal is to design a feedback control law u t = u(p t - T ) that achieves the deterministic convergence of p t to 
a prescribed target state. This problem was solved in [8], [10], [13], for the case of no delay. Note that controlled 
filter equation © shows a significant dependence on the delay, through the input u t = u(p t - T ). Therefore the 
control problem is much more difficult than the previous one. 

B. Delay -dependent stability criteria 

The system of Eq. Q is described by p t G C NxN . By concatenating the real and imaginary part of all elements 
of p t into a column vector, we can rewrite Eq. (0) as a R n -valued nonlinear stochastic delay system. It is important 
to note that the resulting system has the following features: 

• The drift and diffusion terms are polynomials in the state variable. 

• The bounded semi- algebraic set determined by S is positively invariant; see also [8, Proposition 1]. 

• The control input, which possibly suffers from delays, is applied only to the drift term. 

We here do not limit our attention to the specifically structured dynamics of Eq. (0), but rather consider a wide 
class of nonlinear stochastic systems with the above properties. A delay-dependent stability criterion is given in 
Theorem 

throughout this section, the symbols x m (resp. x m ) are used to represent functions (resp. vectors). These symbols with (resp. without) the 
time index denote the solution to Eq. |6j (resp. any functions or vectors). 



Apt = i[u t F v , p t ]dt - - [F z , [F z , p t ]]dt 




(5) 
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Theorem 1: Let /(•,•) : R n x R n — > R n , #(•) : R n — > R n , be polynomials and C a bounded semi- algebraic set 
in R n such that for any initial condition x\ G C£ the solution to the delay differential stochastic equation 



dx t = f(x t ,xt- T )dt + g(xt)dwt 
x Q = x±{0) G C, 9g[-t,0] 



(6) 
(7) 



does not exit C almost surely. Suppose there exist a polynomial K(-) which is positive in C, n-variable polynomials 
Vi (i = 0, 1), S G R 2nxn , and positive-definite matrices R, T e R nxn such that T defined below is negative in 

C x C x R 2n : 

F(x,x d ) := (^) T /(x,x d ) 

- ^i(^d) + V*(x)+T\\g(x)\\*, 



+2 



5(x - x d ) +T\\f(x,x d ) 



\l 



(8) 



T(a;,x d ,2/) := F(x,a; d ) 





T 





S 










-T 





y 









-tR 



X 

x d 

y 

Then, V*(xt) converges to almost surely for any initial condition x\ G Cq. 

Suppose that V*(x) represents a distance between x and a given target state. Then, this theorem states that 
xt converges to the target state if a semi-algebraic problem is feasible; see also Subsection III.C. Semi- algebraic 
problems are in general NP-hard. However, if the degrees of polynomials have been decided, sums of squares (SOS) 
relaxation enables us to solve the problem efficiently [31], [32]. In the numerical example in the next subsection, 
we utilized MATLAB SOSTOOLS [33], [34]. 

Remark 1: In Theorem [TJ T is required to be negative only in C x C x R 2n , not globally (i.e., in R 4n ). 
This is the reason why Theorem 1 can incorporate the structure of density matrices which is useful for reducing 
the conservativeness. Similar criteria for some modified problem formulations (i.e., time- varying delay or delay- 
independent stability) can be obtained straightforwardly. ■ 

We prove Theorem \T\ by using the following Lyapunov-Krasovskii type argument: 

Proposition 1: Let x t be the solution of the stochastic delay differential equations © and 0. Define 

x t (6) :=x t+e , G [-2r,0] 
for t > r. Suppose that there exists a positive functional V defined in Cq t such that 

E[fi/V(x t )+V*(x t )]<0 (9) 
for any t >r. Then, V*(x t ) converges to in the same sense as in Theorem [T] 
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Proof: Recall that x t evolves only in the bounded domain C. Hence Fubini's theorem yields 



E 



J */V(x 8 )ds = J E[s/V(. 



x,)] ds. 



By combining this equality, Eq. ©, and Dynkin's formula [20], [36], we obtain 



E[V(x t )]-V(x T ) 



E 



[/: 



x s )ds 



< 



E[V*(x t )]ds<0. 



Therefore we conclude that V(x t ) is a nonnegative super-martingale. The remainder of the proof is the same as 
the standard Lyapunov-Krasovsii argument; see e.g. Theorems 6.1 and 6.2 in [36] and their proofs. This completes 
the proof. ■ 
Now we are ready to prove Theorem 1. 

Proof of Theorem 1: It suffices to show that V 



V(x) := V o (x(0)) 
r r 



-I I {\\nm,x(- 

J —tJ V 



(10) 



Vi(£(0))d0 

r + e))\\ 2 R +\\g(x(e))\\T}dOdv 

satisfies the assumptions made in Proposition 1. 

The polynomials Vi (i = 0, 1) are bounded from below on C due to the continuity of polynomials and the 
boundedness of the domain. Note that adding any constant to Vi does not affect T. Therefore, without loss of 
generality we can assume that V is positive. 

A direct computation yields 

r° 

< re T Xe - / e T Xeds 
= (2 - 2) • e T S \ x(0) - x(-t) - 



f( 8 )ds 

< 2e T S(x(0) - x(-t)) - J 2e T Sl(s)ds 



-^e J ST- 1 S J e 



x(0) - x{-t) - J f( 3 )di 



where e := 



x(0) T x(-t) t , f(s) := f(x(s),x(-T + s)), and X := SR^S 7 > 0. Combining these 



inequalities and 



£/V(x) 



dVo(x) 



\ l (^) T 



/(x(0),i(-r)) 



g(x(0)) 



+V 1 (x(0)) - V^(-r)) + r(||/(0)|| 2 fi + \\g(x(0))\\ 2 T ) 



{\\m\\ 2 R +\\9(x(s))\\ 2 T}ds, 
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we obtain 



with 



s*V{x) + V*(x(0)) < T(x(0),5(-r)) - Gi(x) - G 2 (x) 



T(x,x d ) := F(x,Xd) 









T 




x J 










x d 





rX+ST-iS 



icT 



G^x) 



e J f(s) T 



ds > 



G 2 (5) := J°Jg(x(s))\\ 2 T ds 



x(0)-x(-r)- / /(s)ds 



> 0. 



X S 
S J R 

Let us take the expectation after substituting x = x t . We can show 

E[G 2 (x t )] = 

by using the ltd isometry. We thus have 

E[fi/V(x t ) + V*(x t )] <E [f (x t ,x t - T ) 
Finally, by the assumption on T and defining 

3>2n 



y ■= 


' T -i ' 




x t 




R- 1 




X t -T 



we obtain 

f(x t ,x t - T ) = T(x u x t - T ,y) < 0. 
Therefore Eq. ([9]) follows. This completes the proof. 



C. Numerical example: Control of a spin- 1/2 system 

This subsection focuses on a spin-1/2 model such that the system is composed of only a single particle. In this 
case, the density matrix p t is in C 2x2 . The filter equation © without the input (i.e., u t = 0) shows the following 
probabilistic convergence: 



pt -> pt := 



l o 
o o 



or p t Pi : = 




1 



This phenomenon is known as quantum state reduction [30]. Here (resp. p\) denotes the eigenstate (of L = F z ) 
for which the monitored spin state of the atom is deterministically up (resp. down). Note that when u t = 0, these 
two matrices are the only equilibrium points of Eq. ([5]). Our goal is to design a feedback control law u t = u(p t - T ) 
that achieves the deterministic convergence of p t to the prescribed target pf, which is either p^ or p^, as we choose. 
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It is shown in [8] that the control input u t = u(p t ) with 

u(p) := /ci(l -tr(pp f )) +fc 2 tr(i[F 2/ ,p]p f ) 



(11) 



achieves the control objective — > Pf when both fci and &2 are chosen appropriatelj^. In this subsection, we derive 
a sufficient condition for this control law to globally stabilize the spin- 1/2 system in the face of feedback delay. 
Let us rewrite Eq. Q in terms of the regulation error 

_ J Pf~ Pu if Pf = P] 
pt-ph ifpf = P|- 

iT 



It can easily be verified that p t — > pi is equivalent to x t := 



0. When we apply the control input 



ut = u(pt-r) with ix(-) given by Eq. (ITTb . the dynamics of are independent of xf and are given by Eq. © with 

— kx d x (2) 



f(x,x d ) := 
flf(a;) := 



^^d (> (1) - |) 



1 «.(2) 

2 



2x (1) (x (1) - 1) 
(2x (1) - l)x (2) 

fcl &2 



Note that pt > means x t is in the circular domain C 



C := 



[ 


x w 







p2 . 



:=x (1) (x (1) -l)+x (2)Z <0V. 



It can be verified that, independently of i^, the solution of Eq. © does not exit C almost surely. In summary, 
according to Theorem 1, if the following SOS decomposition problem has a solution, then the control objective is 
achieved: 

Problem 1: With the definitions above, let V*(x) := ||x|| 2 . Then, find S G M 4x2 , positive-definite matrices 
R, T G M 2x2 , and polynomials V* (i = 0, 1), ft, ft d such that 

-T(x, x d , y) - ft (as, x d , y)V(x) - ft d (a, x d , i/)^(x d ), 

h{x,x d ,y), 

h d (x,x d ,y) 

are the sum of squares of polynomials in x, x d G M 2 and ?/ G M 4 . 

We provide a numerical example to illustrate the effectiveness of Theorem [T] Decision polynomials are restricted 
to quadratic functions. Let k\ = 1.0 and hi = 4.0 which gives the control law whose stabilizing effect for the 

2 The interpretation of this control law is as follows. The second term (containing &2 > 0) locally stabilizes pf. Unfortunately, both pj and 
Pl are equilibria of the closed-loop system. Hence, when pt is close to the eigenstate that is not the regulation point, pt must be prevented 
from converging to it. This is done by the first term. See [35] for a discussion on the effect of delays when a switching control law is employed 
instead. 
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Fig. 2. Time responses of sample paths (thin blue lines) and their average process (thick red line). 

delay-free case was examined in [8, subsection IV. G]. Other parameters are chosen to be r] = 0.9 and r = 0.3. In 
this case, Problem \T\ has a solution; that is, the target state in the controlled system is shown to be stable. It took 
3.01 seconds to check the feasibility of Problem 1 using a computer with a Pentium 4 3.2GHz processor and 2 GB 
memory. 

By setting the target state pi := p^, we performed a numerical simulation. Time responses of the function 

dist(p) := 1 - tr(ppf) : S -> [0, 1] 

are shown in Figure [2] (30 sample paths and their average). This function gives the distance from the target state, 
i.e., dist(p) = (resp. dist(p) = 1) if and only if p = pi (resp. p = p\). The initial state is given by p t = p± for 
— r < t < 0. From Figure [2] it can be seen that stability is achieved. 

Remark 2: In principle, the numerical approach introduced in this subsection is applicable to the stability 
analysis of the general multi-spin system despite time-delays. The computational complexity grows quickly with 
the dimension. Very high dimensional problems are therefore computationally intractable. On the other hand, 
there exist some analytical results for the iV-dimensional delay-free case [10], [28]. The authors are currently 
investigating computational approaches which combine the aforementioned numerical and analytical methods, in 
order to overcome this computational issue. ■ 

IV. CONCLUSION 

From a practical point of view, filter-based quantum control problems should be formulated taking feedback delay 
into explicit account. A delay-dependent stability criterion was derived for a class of nonlinear stochastic systems 
including some quantum spin control systems. A semi- algebraic approach was shown to be useful for incorporating 
the structure of density matrices. 
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Theorem 1 was motivated by quantum spin control systems. Theorem[T]can deal with any stochastic delay system 
having the three properties listed above it. Many finite-dimensional quantum systems satisfy these properties. Hence 
Theorem [T] is applicable to a wide class of finite-dimensional quantum systems. 

This paper is a first attempt to analyze quantum systems which suffer from feedback delays. Hence, many 
important and interesting problems are left unsolved. The research topic mentioned in Remark 2 is one of them. 
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